Cutting MapReduce Cost with Spot Market
نویسنده
چکیده
Spot market provides the ideal mechanism to leverage idle CPU resources and smooth out the computation demands. Unfortunately, few applications can take advantage of spot market because they cannot handle sudden terminations. We describe Spot Cloud MapReduce, the first MapReduce implementation that can fully take advantage of a spot market. Even if a massive number of nodes are terminated regularly due to a price increase, Spot Cloud MapReduce can still make computation progress. We show experimentally that it performs well and it has very little overhead.
منابع مشابه
See Spot Run: Using Spot Instances for MapReduce Workflows
MapReduce is a scalable and fault tolerant framework, patented by Google, for computing embarrassingly parallel reductions. Hadoop is an open-source implementation of Google MapReduce that is made available as a web service to cloud users by the Amazon Web Services (AWS) cloud computing infrastructure. Amazon Spot Instances (SIs) provide an inexpensive yet transient and market-based option to p...
متن کاملReactive Power Pricing Simultaneous Using Spot and Bilateral Market Models Considering Opportunity Cost
Reactive power as a utility of ancillary service in restructured environment is supplied by Independent System Operator (ISO). Due to the particular importance of optimal pricing strategy in the power market, the study aims to investigate this problem more closely. To this end, first the problems of restructuring, reactive power generation and its associated costs thereof were reviewed and diff...
متن کاملBidding Strategy in Spot Markets with Definition of a New Market Power Index by Using Conjectural Variation
In this paper, the concept of Conjectural Variation (CV) is used to specify optimal generation decision for generation companies (Gencos). The conjecture of Genco is defined as its belief or expectation about the reaction of rivals to change of its output. Using CV method, each Genco has to learn and estimate strategic behaviors of other competitors from available historical market operation da...
متن کاملOmni Graph Mining: Graph mining using RDBMS
In this study, we compare three approaches to implement graph algorithms SQL, MapReduce, and GraphChi in terms of performance and cost efficiency. Since MapReduce was introduced, it has been considered as a standard tool to analyze large scale data including graph data. However, we believe that there is a ‘sweet spot’ in which RDBMS can outperform MapReduce. We think that the performance of the...
متن کاملDesign, Development and Testing of Low Capital and Operational Cost Shrub Cutting Machine
The machine to be designed and developed with a view to resolve one ofthe irksome problems of this institute has been facing every year. This institute issurrounded by vast non- fertile land area of about 200 acres, which becomes widelyoccupied by numerous breads of bushes and shrubs during post-rainy season. Due tothis unwanted proliferation of such verdant institute gets abysmal look and that...
متن کامل